Ill posed problems

The example of weighing various combinations of peaches, apples and bananas vividly demonstrates the useful nature of linear systems and the power of linear algebra to solve these systems. You might ask the question: does it matter what combinations of peaches, apples and bananas that I weigh or will this process work with any combination? Consider the following three combinations:

Two peaches, three apples and one banana weigh two pounds
Four peaches, six apples and two bananas weigh four pounds
One peach, four apples, and three bananas weigh seven pounds

The system is modeled mathematically as

Looking carefully at the first two equations, we notice that the second is an exact multiple of the first. Therefore, if you apply Gaussian elimination to this system of equations, multiplying the first equation by 2 and subtracting it from the second equation and replacing the second equation with the result, the second equation disappears!

This is because the first and second equations represent redundant information. Common sense tells you that the second bag of fruit, with exactly twice as many of each variety, and that weighs exactly twice as much, is not telling you anything more than what the first bag of fruit tells you. Systems of equations with this property cannot be solved for unique values of the variables (in this case the weights of individual peaches, apples and bananas).

Suppose the system of equations instead was the following.

In this case, the coefficients on the left hand side of equation two are exactly twice the values in equation one, but the right hand side is not a factor of 2 greater. If you apply Gaussian elimination to these two equations, multiply equation one by 2 and subtract it from equation two and replace equation two with the result, then the left hand side will again disappear, but the right hand side will have a value of . So the resulting second equation will be

In this case the information represented by the first two equations is said to be inconsistent. In either case, redundant or inconsistent, the system of equations cannot be solved. Notice that the clue to either redundant or inconsistent systems is found in the matrix of coefficients on the left hand side of the equation.

If any two rows of the matrix are multiples of one another such that the ratios of corresponding coefficients are all the same value, then the system of equations is either redundant or inconsistent and cannot be solved. Such matrices are said to be singular matrices.

Non-singular matrices have the opposite property; there are as many equations as unknowns and the ratio of any two rows of coefficients does not yield equal ratios for each set of coefficients. Note that if this is the case, then the values of the right hand sides of the equations do not influence the possibility of finding a solution. A solution can be guaranteed for any values on the right hand side of a matrix equation with a non-singular matrix.

The modeling implications of a singular matrix are that you have not modeled the problem in such a way that you have N unique pieces of information (equations). This is extremely important from an engineering analysis point of view.